Testing Linear Models on Ability Parameters

In multiple group item response theory (IRT) modeling, the usual approach to compare the mean abilities in different groups is to introduce normal distributions for every group g, which are indexed by their mean μg and their variance σg. These parameters can be estimated by marginal maximum likelihood (MML, see, for instance, Bock & Aitkin, 1981). Tests of hypotheses concerning group means can be based on the MML estimates of these means divided by their standard errors. In this study we take this approach one step further. We model the mean proficiency levels as a linear combination of structural parameters δ. That is, we consider a model μ=Aδ for some known matrix A. This model allows us to investigate the main and interaction effects of different factors, e.g. gender, place of residence, etc. In this study, the linear model is applied in the framework of the 2and 3-parameter logistic model. For hypothesis testing, we use the likelihood ratio test, the Wald test, and the Lagrange multiplier test. All these tests are used for testing a special linear model against a more general alternative. A simulation study is conducted to assess the small sample Type I error rate and the power as a function of test length, sample size, and effect size. Reference Bock, R.D., & Aitkin, M. (1981). Marginal maximum Likelihood estimation of item parameters: an application of an EM-algorithm. Psychometrika, 46, 443-459.